Human-centric robot with noncontact measurement device

ABSTRACT

A system measuring an object with a robot is provided. The robot including a movable end effector, the robot including a plurality of transducers arranged to transmit signals to an electronic circuit, the electronic circuit configured in operation to determine the position and orientation of the end effector. At least one tool is provided that is removably coupled to the end effector. A three-dimensional (3D) scanner is provided that is configured in operation to determine three-dimensional coordinates of a surface of an object, the 3D scanner being removably coupled to the end effector. A controller is configured to selectively couple one of the at least one tool or the 3D scanner to the end effector in response to an object signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S.application Ser. No. 14/886,135 filed on Oct. 19, 2015, which is aNonprovisional Application of U.S. Provisional Application Ser. No.62/077,513 filed on Nov. 10, 2014, the contents of both of which areincorporated herein by reference.

BACKGROUND

The subject matter disclosed herein relates to a human friendly orhuman-centric robot for use in a manufacturing environment, and inparticular to a human-centric robot that is configured to inspectobjects to determine conformance with predetermined characteristics.

Robotic devices have been widely used in manufacturing and otherenvironments to reduce costs and improve quality. Robotic devices arehard/rigid bodies that may move in a rapid and unpredictable manner. Toavoid unintended impact with human operators, a typical manufacturingcell includes a lock-out procedure whereby the robot device is disabledwhen human operators need to enter the area. By locking out the roboticdevice it is ensured that the risk of impact by a moving robot iseliminated.

One type of robotic device has been developed, referred to as ahuman-centric robot, which allows the robot and the human operator towork in close proximity to each other while minimizing the risk ofimpact to the human operator. These human-centric robots have beenproposed and used in a variety of applications, including medicalfacilities, libraries and manufacturing assembly operations.Human-centric robots include sensors that allow them to monitor theirsurrounding area including the presence of humans. The robot'scontroller is programmed to receive these sensor inputs and predict therisk of impact with nearby humans. When a potential impact on a human isdetected, the robot takes mitigating actions (e.g. slowing down orchanging direction) to avoid contact. In manufacturing environments,these human-centric robots have found use in light assembly and smallpart manufacturing.

Accordingly, while existing robotic devices used in manufacturingenvironments are suitable for their intended purpose the need forimprovement remains, particularly in providing a human-centric robotthat is capable of operating in close proximity to a human operator andalso allows for automated inspection of an object.

BRIEF DESCRIPTION

According to one aspect of the invention, a system is provided. Thesystem includes a robot. The robot having a movable end effectorconfigured to couple with a plurality of tools, the robot including aplurality of transducers arranged to transmit signals to an electroniccircuit, the electronic circuit configured in operation to determine aposition and orientation of the end effector, the robot furtherincluding at least one sensor configured to transmit a position signalfor determining a position of a human operator, the robot configured tooperate directly adjacent to a human operator based at least in part onthe position signal. The system further having at least one tool that isremovably coupled to the end effector. A three-dimensional (3D) scanneris configured in operation to determine three-dimensional coordinates ofa surface of an object, the 3D scanner being removably coupled to theend effector. The system includes a controller having a processor, theprocessor configured to execute computer executable instructions whenexecuted on the processor for selectively coupling one of the at leastone tool or the 3D scanner to the end effector in response to an objectsignal.

According to another aspect of the invention, a method of operating anmanufacturing cell is provided. The method comprising: providing a robotconfigured to operate directly adjacent a human operator, thehuman-centric robot having a movable end effector and a plurality oftransducers arranged to transmit signals to an electric circuit, therobot further including at least one sensor configured to transmit aposition signal for determining a position of a human operator, therobot configured to operate directly adjacent a human operator based atleast in part on the position signal; providing at least one tool;providing a three-dimensional (3D) scanner; receiving an object signal;coupling the at least one tool or 3D scanner to the end effector inresponse to receiving the object signal; performing a first operation onan object being assembled with at least one of the end effector or atleast one tool; and determining the three-dimensional coordinates of atleast one feature of the object with the 3D scanner coupled to the endeffector.

According to still another aspect of the invention, a system forinspecting an object is provided where the object has at least onemachine readable code associated therewith. The system includes a robot.The robot including an articulated arm having at least two arm segmentsand an end effector coupled to the end of the articulated arm. The endeffector is configured to couple with a plurality of tools. Thearticulated arm having a plurality of transducers arranged to transmitsignals to an electronic circuit, the electronic circuit configured inoperation to determine a position and orientation of the end effector.The robot includes at least one sensor configured to detect the positionof an adjacent a human operator. The system further includes a readercircuit operably coupled to the end effector, the reader circuitconfigured to acquire the machine readable code. At least one tool isprovided that is removably coupled to the end effector. A structuredlight three-dimensional (3D) scanner is configured in operation todetermine three-dimensional coordinates of a surface of an object, the3D scanner being removably coupled to the end effector. A controller isprovided having a processor, the processor configured to executecomputer executable instructions when executed on the processor forselectively coupling one of the at least one tool or the 3D scanner tothe end effector in response to acquiring the machine readable code.

These and other advantages and features will become more apparent fromthe following description taken in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter, which is regarded as the invention, is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features, and advantages ofthe invention are apparent from the following detailed description takenin conjunction with the accompanying drawings in which:

FIG. 1 is an illustration of a manufacturing cell having a human-centricrobot in accordance with an embodiment of the invention;

FIG. 2 is an illustration of the manufacturing cell of FIG. 1 with anoncontact measurement device being operated by the human-centric robot;

FIG. 3 is a schematic illustration of a noncontact measurement devicefor use with the human-centric robot of FIG. 1;

FIG. 4 and FIG. 5 are schematic illustrations of another noncontactmeasurement device for use with the human-centric robot of FIG. 1; and

FIG. 6 is a flow diagram of a method of operating the human-centricrobot of FIG. 1.

The detailed description explains embodiments of the invention, togetherwith advantages and features, by way of example with reference to thedrawings.

DETAILED DESCRIPTION

Embodiments of the present invention provide advantages in providingautomated inspection of objects being manufactured in an environmentwhere human operators and robots operate in close proximity to eachother. Embodiments of the present invention allow for cost effectiveinspection of objects being made or fabricated earlier in themanufacturing process to allow errors to be addressed earlier in themanufacturing process to reduce the cost of scrap and rework of theobjects.

Referring now to FIG. 1, a manufacturing cell 20 is illustrated inaccordance with an embodiment. The cell 20 includes a work surface 22,such as a benchtop or a conveyor for example. In the exemplaryembodiment, the cell 20 includes a robotic device 24, such as ahuman-centric robot for example. As used herein, a human-centric robot24 is a robotic device that is configured to operate autonomously orsemi-autonomously in close proximity to a human operator 26. As usedherein, the phrase “close proximity” means that the human-centric robot24 and the operator 26 are positioned such that portions of thehuman-centric robot 24 may move within areas that overlap with the humanoperator 26 during operations. As such, the human-centric robot 24 mayinclude sensors 28, 30 that allow the human-centric robot 24 todetermine if the operator 26 is within a predetermined distance of amoving part of the human-centric robot such that there is a risk ofcontact. A controller 32 is configured to alter the speed or movement ofthe human-centric robot 24 to either avoid contact or reduce the impacton the human operator 26 in the event of contact. In an embodiment, thehuman-centric robot is configured to have a velocity at the point ofcontact of less than or equal to 25 meters/second, a maximum dynamicpower of less than or equal to 80 Watts, or a maximum static force ofless than or equal to 150 Newtons. The sensors 28, 30 may include avisual sensor 28 (e.g. stereoscopic cameras) that is capable of scanningan area or a localized sensor 30, such as a load cell or proximitysensor for example. These sensors transmit a position signal that allowsthe determination of the position of the human operator. In oneembodiment, the human-centric robot may incorporate the characteristicsfor inherent safety described in the journal article “A New ActuationApproach for Human-centric Robot Design” by Zinn et al. (Int. J ofRobotics Research, Vol. 23, No. 4-5, April-May 2004, pp. 379-398), thecontent of which is incorporated herein by reference. In anotherembodiment, the human-centric robot may include the characteristicsdescribed in journal article “Safety Evaluation of Physical Human-RobotInteraction via Crash Testing” by Haddadin et al. (Pro. of Robotics:Science and Systems III, June 2007), the content of which isincorporated herein by reference. In another embodiment, thehuman-centric robot may comply with ISO Standard ISO/DTS 15066, ISO/TR13482:2014 or ISO 10218 for example, the contents of which areincorporated by reference herein.

It should be appreciated that the human-centric robot 24 may alsoinclude additional features, such as bumpers or padding for example,that reduces the risks associated with a robot operating autonomously orsemi-autonomously in the proximity of a human operator. In theillustrated embodiment, the human-centric robot 24 includes a base 31with an articulated arm 34 have two or more arm segments 36, 38.Arranged at the end of the arm 34 is an end effector 40. As will bediscussed in more detail herein, in the exemplary embodiment the endeffector 40 includes a coupler 42 that is sized to accept a tool. In oneembodiment, the end effector 40 includes a reader circuit 44. The readercircuit 44 is configured to receive or acquire a machine readable code,such an optical code (e.g. bar code) or an electromagnetic radiationcode (e.g. near field communication signal, radio frequencyidentification signal (RFID), WiFi signal or Bluetooth signal). Wherethe reader circuit 44 is configured to receive an optical code, thereader circuit 44 may include an optical scanner configured to transmita light and acquire an image of the machine readable code. In oneembodiment, the machine readable code is transmitted audibly, such as bya spoken word of the human operator for example. In this embodiment, thereader circuit 44 may include a microphone.

The arm 34 further includes a plurality of transducers 46. In theexemplary embodiment, each one of the transducers 46 is associated withone of the axis of rotation of the arm segments 36, 38 to measure therotation of the arm segment. The transducer 46 may be a rotary encoderfor example. Each of the transducers 46 is electrically coupled totransmit a signal to the controller 32 in response to the rotation ofthe associated arm segment. In this manner, the controller 32 maydetermine the position and orientation of the end effector 40 and anyportion of the arm 34.

It should be appreciated that while embodiments herein illustrate thehuman-centric robot as being stationary with a multi-axis articulatedarm, this is for exemplary purposes and the claimed invention should notbe so limited. In other embodiments, the human-centric robot may bemobile, have multiple articulated arms, have multiple actuators/couplersor a combination thereof for example.

Controller 32 is a suitable electronic device capable of accepting dataand instructions, executing the instructions to process the data, andpresenting the results. Controller 32 may accept instructions throughuser interface, or through other means such as but not limited toelectronic data card, voice activation means, manually-operableselection and control means, radiated wavelength and electronic orelectrical transfer. Therefore, controller 32 can be a microprocessor,microcomputer, a minicomputer, an optical computer, a board computer, acomplex instruction set computer, an application specific integratedcircuit (ASIC), a reduced instruction set computer (RSIC), an analogcomputer, a digital computer, a molecular computer, a quantum computer,a cellular computer, a superconducting computer, a supercomputer, asolid-state computer, a single-board computer, a buffered computer, acomputer network, a desktop computer, a laptop computer, a scientificcomputer, a cellular phone or a hybrid of any of the foregoing.

Controller 32 is capable of converting the analog voltage level providedby sensors 28, 30, encoders 46 and reader circuit 44 into a digitalsignal. In one embodiment, the sensors 28, 30, encoders 46 or readercircuit 44 may be configured to provide a digital signal to controller32, or an analog-to-digital (A/D) converter (not shown) maybe coupledbetween sensors 28, 30, encoders 46 or reader circuit 44 and controller32 to convert the analog signal into a digital signal for processing bycontroller 32. In other embodiments, the signals may be transferredbetween the sensors 28, 30, encoders 46, reader circuit 44 andcontroller 32 by fiber optic cables. Controller 32 uses the digitalsignals act as input to various processes for controlling thehuman-centric robot 24. The digital signals represent one or morehuman-centric robot data including but not limited to the proximitydistances to the human operator, machine readable code and arm segmentrotation for example.

In general, controller 32 accepts data and is given certain instructionsfor the purpose of comparing the data to predetermined parameters. Thecontroller 32 compares the parameters to predetermined variances (e.g.the arm 34 is approaching the human operator) and if the predeterminedvariance is exceeded may generate a signal that may be used to changethe operation of the human-centric robot or indicate an alarm to thehuman operator. In one embodiment, the controller 32 may be configuredto transmit an alert signal to a remote computer (not shown) or totransmit a signal via another communications medium, such as a cellularSMS (text message) signal to a predetermined third party for example.

The controller 32 may include an electronic circuit. The electroniccircuit may include a processor 48 coupled to one or more memory devices50. The memory devices 50 may include random access memory (RAM) device,a non-volatile memory (NVM) device or a read-only memory (ROM) device.The processor 48 may also be coupled to one or more input/output (I/O)controllers and a LAN interface device via a data communications bus.

The memory devices store an application code, e.g., main functionalityfirmware, including initializing parameters, and boot code, for theprocessor. Application code also includes program instructions forcausing processor to execute any operation control methods, includingstarting and stopping operation, and determining the probability of thearm 34 contacting the human operator 26, based on the output voltagesignal, and generation of alarms. The application code may create anonboard telemetry system may be used to transmit operating informationbetween the human-centric robot 24 and a remote terminal location andor/receiving location. As will be discussed in more detail below, theoperation control methods may include measuring coordinates or points onan object 56 being worked on in the cell 20.

It should be appreciated that the controller 32 may be remotely locatedfrom the base 31. In an embodiment, the human-centric robot 24 mayinclude a communications circuit (e.g. WiFi, Bluetooth, cellular,Ethernet) that transmits the output voltage signal to the remotelylocated controller 36. In one embodiment, the controller 32 may be acellular phone that connects to the human-centric robot 24 via a wiredor wireless communications medium.

In the exemplary embodiment, the human-centric robot 24 includes a toolmagazine 50 arranged to receive and store tools 52 and noncontactmeasurement device 54. The tool magazine 50 includes a plurality of toolholders that are similarly configured to receive the shank or grippingportion of a tool 52 and noncontact measurement device 54. The tools 52and noncontact measurement device 54 may be selectively transferred bythe controller 32 between the tool magazine 50 and the coupler 42automatically during operation, such as in response to a signal fromreader circuit 44 for example.

It should be appreciated what while the tool magazine 50 is illustratedwith the holders extending perpendicular to the tool magazine 50, thisis for exemplary purposes and other tool magazine and tool holderconfigurations are possible. For example, the tool magazine may haveholders that extend radially from the outer diameter/periphery of thetool magazine. In another embodiment, the tool magazine may include aconveyor type system that follows a serpentine path. Further, while thetool magazine 50 is illustrated as being mounted directly adjacent thehuman-centric robot 24, in other embodiments, the tool magazine may beremotely located and may be retrieved by a separate robotic device.Further, the tool magazine may be remotely located in an enclosure thatmay be selectively isolated (e.g. with a movable door) to shield thetool magazine and the tools stored therein from debris, cooling fluidand lubricants used during the manufacturing process within cell 20.

In one embodiment, the manufacturing assembly line 20 receives an object56 that will be processed by the human-centric robot 24 or the humanoperator 26. It should be appreciated that while the object 56 isillustrated as a single item, this is for exemplary purposes and theobject 56 may be comprised of a plurality of components or subassembliesthat may require assembly by the human-centric robot 24 or the humanoperator 26. The plurality or collection of components or subassembliesmay be arranged in a kit for example. The object 56 may also requirefabrication process steps, such as but not limited to machining,welding, bonding for example. The object 56 may also require chemicalprocessing for example.

The object 56 may include an associated communications module 58. Aswill be discussed in more detail below, the communications module 58 mayinclude one or more sub-modules, such as a near field communicationscircuit (NFC), a cellular teleconference circuit (including LTE, GSM,EDGE, UMTS, HSPA and 3GPP cellular network technologies), a Bluetooth®(IEEE 802.15.1 and its successors) circuit and a Wi-Fi (IEEE 802.12)circuit for example. For exemplary purposes, the communications module58 will be described in reference to an NFC communications module,however, it should be appreciated that the communications module 58 mayincorporate any of the foregoing communications protocols or acombination thereof.

The NFC communications module 58 may be a passive device, meaning thatthe communications module stores information that is transmitted inresponse to an external radio frequency signal. In one embodiment, theNFC communications module 58 is a single port NFC tag, such as MIFAREClassic Series manufactured by NXP Semiconductors for example. The NFCcommunications module 58 stores data regarding the object 56, such asbut not limited to: manufacturing process instructions, inspectioninstructions, object 56 identification information (e.g. part number),and dimensional data information from preceding manufacturing processes.

It should be appreciated that in other embodiments, other communicationsprotocols may be used rather than NFC. In another embodiment, a radiofrequency identification (RFID) protocol is used. An RFID module or RFID“tag” operates in a similar manner to NFC communications module exceptthat the RFID module may operate at a longer or farther distance fromthe reader circuit. The range for a passive RFID system is typicallyreadable up to 82 feet (25 meters), while an NFC system is typicallyreadable at a range of less than 10 inches (25 centimeters) and in someembodiments NFC systems have an operating range of several inches.

In one embodiment, the NFC communications module 58 is a dual-interfacememory/tag device such as the M24SR series NFC tags manufactured by STMicroelectronics N.V. for example. A dual-interface memory deviceincludes a wireless port that communicates with an external NFC reader,and a wired port that connects the device with another circuit. Itshould be appreciated that the NFC communications module 58 maysometimes colloquially be referred to as a “tag.”

As will be discussed in more detail below, the human-centric robot 24may be configured to perform one or more noncontact measurements of theobject 56. In one embodiment, the NFC communications module dataacquired by these noncontact measurements may be transmitted from thereader circuit 44 to the NFC communications module 58. In oneembodiment, this noncontact measurement data may be transmitted from theNFC communications module 58 to a processing circuit associated with theobject 56. It should be appreciated that this may provide advantages inreducing costs and increasing reliability, such as by allowingprocessing circuit of the object 56 to use the communicated measurementdata for calibration of the object without intervention from the humanoperator for example. In one embodiment, a calibration member 57 may bemounted on or located near the work surface 22. The calibration member57 may be a card with a pattern, such as a checker board for example, ora three-dimensional object having known parameters for example.

The human-centric robot 24 is configured to selectively remove and storetools 52 and measurement devices 54 in the tool magazine 50. Theselection of the tool 52 or measurement device 54 may be autonomously orsemi-autonomously performed by the human-centric robot 24, such as inresponse to a signal from the NFC communications module 58 for example.In one embodiment, shown in FIG. 2, the human-centric robot 24 mayselect the noncontact measurement device 54 and couple the stem orgripper portion to the coupler 42. The human-centric robot 24 then movesthe arm 34 to position the noncontact measurement device 54 adjacent tothe object 56 to perform one or more predetermined measurements. Asdiscussed above the measurements may be defined for the human-centricrobot 24 by manufacturing process instruction data or inspection datatransmitted by the NFC communications module 58 for example.

It should be appreciated that the human operator 26 may be performingother operations on the object 56 in close proximity to the arm 34 whilethe measurements are being acquired by the noncontact measurement device54. In one embodiment, the human-centric robot 24 may include a userinterface, such as a display or monitor for example, that providesinformation such as the measurement data for example to the humanoperator 26 in real-time or in near real-time. This provides advantagesin allowing the human operator 26 to make changes or adjustments to theobject 56, such as for calibration purposes or to adjust the object'sperformance for example.

In one embodiment, the user interface may be remote from thehuman-centric robot 24, such as application software executed on amobile device (e.g. a cellular phone, tablet, laptop or wearablecomputing device). The wearable device may be, for example, glasseshaving a display that shows the user the data/information from thehuman-centric robot 24 as described herein. The wearable device may alsobe a watch with a display that shows the user data/information from thehuman-centric robot 24. The wearable device may further be an articlesuch as a badge, ring, broach or pendant, that either displaysinformation from the human-centric robot 24. It should be appreciatedthat these wearable devices may also indicate or display a subset of thedata/information, for example, a ring may have an indicator that changescolor based on a measurement parameter (e.g. the measurement wassuccessfully acquired). The wearable device and other portable computingdevices each have a processor and memory that is configured to executecomputer instructions on the respective processor to perform thefunctions described herein. In still another embodiment, the userinterface may be an audio speaker operably coupled to the controller 32that allows the human-centric robot 24 to communicate information orinstructions to the human operator 26 using audibly, such as with spoken(e.g. English) words or audible tone codes.

In the exemplary embodiment, the noncontact measurement device 54 may bea laser line probe (FIG. 3) or a structured light scanner (FIG. 4 andFIG. 5). Referring now to FIG. 3, a noncontact measurement device 54that is a laser line probe 100 will be described. The line scanner 100includes a projector 102 and a camera 104, the camera including a lenssystem 106 and a photosensitive array 108 and the projector including anobjective lens system 110 and a pattern generator 112. The projector 102projects a line 114 (shown in the figure as projecting out of the planeof the paper) onto the surface 116 of an object 56, which may be placedat a first position 118 or a second position 120. Light scattered fromthe object 56 at the first point 122 travels through a perspectivecenter 124 of the lens system 106 to arrive at the photosensitive array108 at position 126. Light scattered from the object at the secondposition 128 travels through the perspective center 124 to arrive atposition 130. By knowing the relative positions and orientations of theprojector 102, the camera lens system 106, the photosensitive array 108,and the position 126 on the photosensitive array, it is possible tocalculate the three-dimensional coordinates of the point 122 on theobject 56 surface. Similarly, knowledge of the relative position of thepoint 130 rather than 126 will yield the three-dimensional coordinatesof the point 128. The photosensitive array 108 may be tilted at an angleto satisfy the Scheimpflug principle, thereby helping to keep the lineof light on the object surface in focus on the array.

One of the calculations described herein above yields information aboutthe distance of the object from the line scanner. In other words, thedistance in the z direction, as indicated by the coordinate system 132.The information about the x position and y position of each point 122 or128 relative to the line scanner is obtained by the other dimension ofthe photosensitive array 108, in other words, the y dimension of thephotosensitive array. Since the plane that defines the line of light asit propagates from the projector 102 to the object is known from theencoder 46 signals of the human-centric robot 24 articulated arm 34, itfollows that the x position of the point 122 or 128 on the objectsurface is also known. Hence all three coordinates—x, y, and z—of apoint on the object surface can be found from the pattern of light onthe two-dimensional array 108.

Referring now to FIG. 4 and FIG. 5, a noncontact measurement device 54that is a structured light scanner 500 will be described. As used hereinthe term “structured light” means a device that projects a patternhaving elements there on. As will be discussed below, the structuredlight pattern may be coded or uncoded.

The scanner 500 first emits a structured light pattern 522 withprojector 508 onto surface 524 of an object 56. The structured lightpattern 522 may include the patterns disclosed in the journal article“DLP-Based Structured Light 3D Imaging Technologies and Applications” byJason Geng published in the Proceedings of SPIE, Vol. 7932, the contentof which is incorporated herein by reference. The light 509 fromprojector 508 is reflected from the surface 524 and the reflected light511 is received by the camera 510. It should be appreciated thatvariations in the surface 524, such as protrusion 526 for example,create distortions in the structured pattern when the image of thepattern is captured by the camera 510. Since the pattern is formed bystructured light, it is possible in some instances for a controller 512to determine a one to one correspondence between the pixels in theemitted pattern, such as pixel 513, for example, and the pixels in theimaged pattern, such as pixel 515 for example. The correspondencebetween the pixels may be performed by a processing circuit 512 withinthe structured light scanner 500, or the acquired data may betransmitted to the controller 32.

Determining the correspondence between pixels enables the use oftriangulation principals in determining the coordinates of each pixel inthe imaged pattern. The collection of three-dimensional coordinates ofthe surface 524 is sometimes referred to as a point cloud. By moving thescanner 500 over the surface 524 with the arm 34, a point cloud may becreated of the object 56.

To determine the coordinates of the pixel, the angle of each projectedray of light 509 intersecting the object 522 in a point 527 is known tocorrespond to a projection angle phi (Φ), so that Φ information isencoded into the emitted pattern. In an embodiment, the system isconfigured to enable the Φ value corresponding to each pixel in theimaged pattern to be ascertained. Further, an angle omega (Ω) for eachpixel in the camera is known, as is the baseline distance “D” betweenthe projector 508 and the camera. Therefore, the distance “Z” from thecamera 510 to the location that the pixel has imaged using the equation:

$\begin{matrix}{\frac{Z}{D} = \frac{\sin \mspace{11mu} (\Phi)}{\sin \mspace{11mu} ( {\Omega + \Phi} )}} & (1)\end{matrix}$

Thus three-dimensional coordinates may be calculated for each pixel inthe acquired image.

In general, there are two categories of structured light, namely codedand uncoded structured light. A common form of uncoded structured lightrelies on a striped pattern varying in a periodic manner along onedimension. These types of patterns are usually applied in a sequence toprovide an approximate distance to the object. Some uncoded patternembodiments, such as the sinusoidal patterns for example, may providerelatively highly accurate measurements. However, for these types ofpatterns to be effective, it is usually necessary for the scanner deviceand the object to be held stationary relative to each other. In oneembodiment, the scanner 500 may use an uncoded pattern where the objector scanner is not stationary. In this embodiment, the scanner 500 may bethe type described in commonly owned U.S. patent application Ser. No.13/767,167 entitled “Device for Optically Scanning and Measuring anEnvironment” filed on Feb. 14, 2013, the content of which isincorporated herein by reference.

Where the scanner device or the object are in motion (relative to theother), then a coded pattern may be desired. A coded pattern allows theimage to be analyzed using a single acquired image. Some coded patternsmay be placed in a particular orientation on the projector pattern (forexample, perpendicular to epipolar lines on the projector plane),thereby simplifying analysis of the three-dimensional surfacecoordinates based on a single image.

Epipolar lines are mathematical lines formed by the intersection ofepipolar planes and the source plane 517 or the image plane 521 (theplane of the camera sensor) in FIG. 5. An epipolar plane may be anyplane that passes through the projector perspective center 519 and thecamera perspective center. The epipolar lines on the source plane 517and the image plane 521 may be parallel in some cases, but in generalare not parallel. An aspect of epipolar lines is that a given epipolarline on the projector plane 517 has a corresponding epipolar line on theimage plane 521. Therefore, any particular pattern known on an epipolarline in the projector plane 517 may be immediately observed andevaluated in the image plane 521. For example, if a coded pattern isplaced along an epipolar line in the projector plane 517, the spacingbetween the coded elements in the image plane 521 may be determinedusing the values read out of the pixels of the camera sensor 510. Thisinformation may be used to determine the three-dimensional coordinatesof a point 527 on the object 56. It is further possible to tilt codedpatterns at a known angle with respect to an epipolar line andefficiently extract object surface coordinates.

In embodiments having a periodic pattern, such as a sinusoidallyrepeating pattern, the sinusoidal period represents a plurality ofpattern elements. Since there is a multiplicity of periodic patterns intwo-dimensions, the pattern elements are non-collinear. In some cases, astriped pattern having stripes of varying width may represent a codedpattern.

Referring now to FIG. 6, a method 600 of operating the manufacturingcell 20 is shown in accordance with one embodiment of the invention. Themethod 600 initiates in block 602 where the human-centric robot 24electronically identifies the object 56. This may be done for example bymoving the arm 34 to place the end effector 40 in close proximity to theobject 56 such that the reader circuit 44. It should be appreciated thatwhen the communications module 58 is within the reader circuit's 44range is an object signal may be transmitted from the communicationmodule 58. As used herein the reader circuit's 44 range is the distanceat which an reader circuit 44 can detect a signal from an externaldevice. In one embodiment, the signal may be in the form of a modulationof the waveform transmitted by the reader circuit 44. As discussedabove, the object signal may include data associated with the object 56,such as but not limited to manufacturing process steps, inspection stepsand product identification for example. The object signal is generatedin block 604. Upon receipt by the controller 32, the controllerdetermines in query block 606 whether a tool 52 or a noncontactmeasurement device 54 is to be used in operation. It should beappreciated that in some embodiments, both a tool 52 and a noncontactmeasurement device 54 may be used with the object 56.

In some instances, a tool 52 will be used first, such as to tighten afastener or drill a hole for example. The method 600 then proceeds toblock 608 where the tool 52 is removed from the tool magazine 50. Thehuman-centric robot 24 then proceeds to perform the operation (e.g.tighten a fastener, couple sub-assemblies) in block 610. Once theoperation is complete, the method 600 stores the tool 52 in magazine 50in block 612. The method 600 then loops back to determine if the nextsequence of the operation uses a tool 52 or a noncontact measurementdevice 54.

If query block 606 determines that a measurement is to be taken of theobject 54, the method 600 proceeds to block 614 where the noncontactmeasurement device 54 is retrieved from the tool magazine 50. In theexemplary embodiment, the human-centric robot 24 couples the stem orgripper portion of the noncontact measurement device to the coupler 42on the end effector 40. The human-centric robot 24 then positions thearm 34 to allow the noncontact measurement device 54 to measure thedesired features of the object 56. In some embodiments, it may bedesirable to perform a compensation process with the noncontactmeasurement device 54 prior to measuring the object 56. The compensationprocess typically includes measuring article or articles or a knownsize, such as but not limited to a plate, a sphere or a plurality oftargets for example.

The method 600 then proceeds to block 616 where the object 56 is scannedand the desired measurements are obtained in block 618. The method 600then proceeds to perform some predetermined analysis. In the exemplaryembodiment, the analysis includes determining if the measurements arewithin a predetermined specification in block 620. The predeterminedspecification value may have been transmitted for example in the datacontained in the object signal. Other types of analysis may also beprovided, such as determining a compensation value or calibrationparameter for the object 56 for example. The method 600 then proceeds toblock 622 where the measurement data or the comparison of themeasurements to the predetermined specification value is transmitted tothe operator 26. It should be appreciated that the method 600 may alsotake other steps, such as transmitted data to the communications module58 for example, or providing an alarm or alert if the measured valuesdeviate from an acceptable criteria.

Finally, the method 600 then loops back to determine if the nextsequence of the operation uses a tool 52 or a noncontact measurementdevice 54. This process continues until the desired sequence of stepsbeing performed by the human-centric robot is completed. It should beappreciated that where the human-centric robot 24 has multiplearticulated arms; these steps may be carried out in parallel forexample. Further, while embodiments refer to a tool magazine, in otherembodiments the tool magazine may be omitted and the human-centric robot24 utilizes sensors, such as sensor 28 for example, to identify thetools 52 and noncontact measurement devices 54 on a work surface. Instill other embodiments, the human-centric robot 24 cooperates with thehuman operator 26 to guide the human operator 26 in the manufacturingsteps being performed on the object 56, such as by using the noncontactmeasurement device 54 to measure the position of a tool 60 being used bythe human operator 26.

While the invention has been described in detail in connection with onlya limited number of embodiments, it should be readily understood thatthe invention is not limited to such disclosed embodiments. Rather, theinvention can be modified to incorporate any number of variations,alterations, substitutions or equivalent arrangements not heretoforedescribed, but which are commensurate with the spirit and scope of theinvention. Additionally, while various embodiments of the invention havebeen described, it is to be understood that aspects of the invention mayinclude only some of the described embodiments. Accordingly, theinvention is not to be seen as limited by the foregoing description, butis only limited by the scope of the appended claims.

1. A system comprising: a robot having a movable end effector, the robotincluding a plurality of transducers arranged to transmit signals to anelectronic circuit, the electronic circuit configured in operation todetermine a position and orientation of the end effector, the robotfurther including at least one sensor configured to transmit a positionsignal for determining a position of a human operator, the robotconfigured to operate directly adjacent to a human operator based atleast in part on the position signal; at least one tool removablycoupled to the end effector; a three-dimensional (3D) scanner configuredin operation to determine three-dimensional coordinates of a surface ofan object, the 3D scanner being removably coupled to the end effector;and a controller having a processor, the processor configured to executecomputer executable instructions when executed on the processor forselectively coupling one of the at least one tool or the 3D scanner tothe end effector in response to an object signal.
 2. The system of claim1, wherein the robot further includes a machine readable optical scannerthat generates in operation a scanner signal in response to scanning abar code.
 3. The system of claim 1, wherein the robot further includes aradio frequency identification (RFID) reader circuit that generates inoperation the object signal in response to an RFID tag being withinrange of the reader circuit.
 4. The system of claim 1, wherein the robotfurther includes a near field communications (NFC) reader circuit thatgenerates in operation the object signal in response to an NFC tag beingwithin range of the reader circuit.
 5. The system of claim 1, whereinthe at least one tool includes a plurality of tools.
 6. The system ofclaim 5, further comprising a tool holder operably coupled to the robot,the tool holder having a plurality of modules for storing the pluralityof tools and the 3D scanner, the tool holder is configured in operationto remove and store the plurality of tools and the 3D scanner duringoperation.
 7. The system of claim 6, further comprising a calibrationmember adjacent the robot, wherein the processor is configured tocalibrate the 3D scanner with the calibration member in response to the3D scanner being coupled to the end effector.
 8. A method of operating amanufacturing cell, the method comprising: providing a configured tooperate directly adjacent a human operator, the robot having a movableend effector and a plurality of transducers arranged to transmit aposition signal to an electric circuit, the robot further including atleast one sensor configured to transmit a position signal fordetermining a position of a human operator, the robot configured tooperate directly adjacent to a human operator based at least in part onthe position signal; providing at least one tool; providing athree-dimensional (3D) scanner; receiving an object signal; coupling theat least one tool or 3D scanner to the end effector in response toreceiving the object signal; performing a first operation on an objectbeing assembled with at least one of the end effector or the at leastone tool; and determining three-dimensional coordinates of at least onefeature of the object with the 3D scanner coupled to the end effector.9. The method of claim 8, further comprising: providing the robot with aradio frequency identification (RFID) reader circuit; and generating acommand signal with the reader circuit in response to the reader circuitbeing within range of an RFID tag.
 10. The method of claim 8, furthercomprising: providing the robot with a near field communications (NFC)reader circuit; and generating the object signal with the reader circuitin response to the reader circuit being within range of an NFC tag. 11.The method of claim 8 wherein the at least one tool includes at least afirst tool and a second tool.
 12. The method of claim 11, furthercomprising: selectively coupling one of the first tool and the secondtool to the end effector in response to a command signal; and performinga second operation on the object with the robot in response to the firsttool or the second tool being coupled to the end effector.
 13. Themethod of claim 8, further comprising: providing a tool holder operablycoupled to the robot, the tool holder having a plurality of modules forstoring the at least one tool and the 3D scanner; removing the 3Dscanner from the tool holder; moving the 3D scanner adjacent the objectprior to determining the three-dimensional coordinates; and storing the3D scanner in the tool holder after determining the three-dimensionalcoordinates.
 14. The method of claim 13, further comprising: providing acalibration member positioned adjacent the, robot; and calibrating the3D scanner with the calibration member prior to moving the 3D scanneradjacent the object.
 15. A system for inspecting an object, the objecthaving at least one machine readable code associated therewith, thesystem comprising: a robot having an articulated arm with at least twoarm segments and an end effector arranged on an end of the articulatedarm, the end effector configured to couple with a plurality of tools,the articulated arm including a plurality of transducers arranged totransmit signals to an electronic circuit, the electronic circuitconfigured in operation to determine a position and orientation of theend effector, the robot further having at least one sensor configured todetect the position of an adjacent a human operator; a reader circuitoperably coupled to the end effector, the reader circuit configured inoperation to acquire the machine readable code; at least one toolremovably coupled to the end effector; a structured lightthree-dimensional (3D) scanner configured in operation to determinethree-dimensional coordinates of a surface of an object, the 3D scannerbeing removably coupled to the end effector; and a controller having aprocessor, the processor configured to execute computer executableinstructions when executed on the processor for selectively coupling oneof the at least one tool or the 3D scanner to the end effector inresponse to acquiring the machine readable code.
 16. The system of claim15, wherein the reader circuit includes a machine readable opticalscanner that generates in operation a scanner signal in response toscanning a bar code.
 17. The system of claim 16, wherein the readercircuit that communicates in operation using a radio frequencyidentification (RFID) protocol or an near-field communication (NFC)protocol.
 18. The system of claim 15, wherein the at least one toolincludes a plurality of tools.
 19. The system of claim 18, furthercomprising a tool holder operably coupled to the robot, the tool holderhaving a plurality of modules for storing the plurality of tools and the3D scanner, the tool holder cooperating in operation with the endeffector to remove and store the plurality of tools and the 3D scanner.20. The system of claim 16, further comprising a calibration memberadjacent the robot, wherein the processor is configured to calibrate the3D scanner with the calibration member in response to the 3D scannerbeing coupled to the end effector.